Automated methods and systems for vascular plaque detection and analysis

ABSTRACT

Automated methods and systems for the detection and analysis of plaque in one or more regions of a patient&#39;s vasculature are described.

RELATED APPLICATIONS

This patent application claims priority to, and the benefit of, each ofthe following patent applications: U.S. provisional patent applicationNo. 60/497,375, filed 21 Aug. 2003; and U.S. non-provisional patentapplication Ser. No. 10/923,124, filed 21 Aug. 2004, each of which ishereby incorporated in its entirety for all purposes.

TECHNICAL FIELD

This invention concerns methods, software, and systems for the automatedanalysis of medical imaging data. Specifically, it concerns methods,software, and systems for the automated detection and analysis of plaquewithin part or all of a patient's vasculature.

BACKGROUND OF THE INVENTION

1. Introduction

The following description includes information that may be useful inunderstanding the present invention. It is not an admission that anysuch information is prior art, or relevant, to the presently claimedinventions, or that any publication specifically or implicitlyreferenced is prior art.

2. Background

Atherosclerosis is the most common cause of ischemic heart disease. Whenconsidered separately, stroke is the third leading cause of death, withthe vast majority of strokes being the result of ischemic events.However, arteriosclerosis is a quite common inflammatory response, andatherosclerosis without thrombosis is in general a benign disease.Several studies indicate that the plaque composition rather than thedegree of stenosis is the key factor for predicting vulnerability torupture or thrombosis. Such thrombosis-prone or high-risk plaques arereferred to as “vulnerable” plaques.

Plaque rupture is triggered by mechanical events, but plaquevulnerability is due to weakening of the fibrous cap, interplaquehemorrhage, and softening of plaque components, often as a result ofinfection and macrophage and T-cell infiltration. In general,lipid-rich, soft plaques are more prone to rupture than collagen-rich,hard plaques. Several morphological and physiological features areassociated with vulnerable and stable plaque. Morphologicalcharacteristics suggest structural weakness or damage (thin or rupturedfibrous cap, calcification, negative remodeling, neovascularization,large lipid deposits, etc.), while physiological features suggestchemical composition, active infection, inflammatory responses, andmetabolism. Many of the factors are subjective or qualitative,reflecting the fact that not all characteristics have been validated asrisk determinants. The validation of risk factors requires long-termlongitudinal clinical studies, endarterectomies, or autopsies.

Several invasive methods have been used to identify vulnerable plaque,including intravenous ultrasound (IVUS), angioscopy, intravascular MR,and thermography. Since invasive methods expose the patient tosignificant risk of stroke and MI, they are not appropriate forscreening or serial examination. Finally, since these methods requirethe use of a catheter, estimates of overall vascular plaque burden mustbe extrapolated from examination of only a few local plaque deposits.Moreover, due to physical constraints such as catheter and artery size,arterial branching, etc., much of a patient's vasculature isinaccessible to invasive instruments.

While MRI has been used to identify morphological plaque features, suchas plaque size and fibrous cap thickness, with high sensitivity andspecificity, most efforts to characterize plaque involve visualinspection of CAT or MRI scans by expert radiologists. This is atime-consuming (and thus expensive) and error-prone process, subject toseveral subjective biases, not least that humans are notoriously poor atsimultaneously assessing statistical relationships between more than twoor three variables. A natural tendency is to focus on gross boundariesand local textures. When considering multimodal images, this problem ismultiplied several-fold because in order to digest all the availableevidence, the analyst has to assess, pixel-by-pixel, the localenvironment in as many as four distinct modalities. Typically, thisforces the analyst to concentrate on only one modality, with the “best”contrast for a particular tissue, and disregard potential contraryevidence in the other modalities. Classification accuracy is subject tovariability between researchers and even for the same researcher overtime, making a standardized diagnostic test virtually impossible. Inmost cases, validation of the interpreted image can only be accomplishedby histological examination of endarterectomies.

Given these importance of plaque detection and analysis to patienthealth, there is a clear need for improved methods for the detection andanalysis of plaque in vivo.

3. Definitions

Before describing the instant invention in detail, several terms used inthe context of the present invention will be defined. In addition tothese terms, others are defined elsewhere in the specification, asnecessary. Unless otherwise expressly defined herein, terms of art usedin this specification will have their art-recognized meanings.

A “medical imaging system” refers to any system that can be used togather, process, and generate images of some or all of the internalregions a patient's body. Typically such systems include a device togenerate and gather data, as well as a computer configured to processand analyze data, and frequently generate output images representing thedata. Devices used to generate and gather data include those that arenon-invasive, e.g., magnetic resonance imaging (“MRI”) machines,positron emission tomography (“PET”) machines, computerized axialtomography (“CAT”) machines, ultrasound machines, etc., as well asdevices that generate and collect data invasively, e.g., endoscopes (fortransmission of visual images from inside a cavity or lumen in the body)and catheters with a sensing capability. Data collected from suchdevices are then transmitted to a processor, which in at least somecases, can be used to produce images of one or more internal regions ofthe patient's body. A healthcare professional trained to interpret theimages then examines and interprets the images to generate a diagnosisor prognosis.

A “patentable” composition, process, machine, article of manufacture, orimprovement according to the invention means that the subject mattersatisfies all statutory requirements for patentability at the time theanalysis is performed. For example, with regard to novelty,non-obviousness, or the like, if later investigation reveals that one ormore claims encompass one or more embodiments that would negate novelty,non-obviousness, etc., the claim(s), being limited by definition to“patentable” embodiments, specifically exclude the unpatentableembodiment(s). Also, the claims appended hereto are to be interpretedboth to provide the broadest reasonable scope, as well as to preservetheir validity. Furthermore, if one or more of the statutoryrequirements for patentability are amended or if the standards changefor assessing whether a particular statutory requirement forpatentability is satisfied from the time this application is filed orissues as a patent to a time the validity of one or more of the appendedclaims is questioned, the claims are to be interpreted in a way that (1)preserves their validity and (2) provides the broadest reasonableinterpretation under the circumstances.

The term “treatment” or “treating” means any treatment of a disease ordisorder, including preventing or protecting against the disease ordisorder (that is, causing the clinical symptoms (or the underlyingprocess that may produce or contribute to the symptoms) not to develop);inhibiting the disease or disorder (i.e., arresting or suppressing thedevelopment of clinical symptoms, or suppressing progression of one ormore underlying process that contributes to the pathology that mayproduce symptoms); and/or relieving the disease or disorder (i.e.,causing the regression of clinical symptoms; or regression of one ormore processes that contribute to the symptoms). As will be appreciated,it is not always possible to distinguish between “preventing” and“suppressing” a disease or disorder since the ultimate inductive eventor events may be unknown or latent. Accordingly, the term “prophylaxis”will be understood to constitute a type of “treatment” that encompasseseither or both “preventing” and/or “suppressing”. The term “protection”thus includes “prophylaxis”.

SUMMARY OF THE INVENTION

It is an object of this invention to provide patentable methods,software, and systems for the automated detection and, if desired,analysis of plaque in one or more regions of a patient's vasculatureobtained from data from a medical imaging system, or the initial sensingor data collection processes such as (but not limited to) those thatcould be used to generate an image.

Thus, in one aspect, the invention concerns automated methods ofassessing a degree of atherosclerosis in at least a portion of apatient's vasculature, frequently in part of all of one or more bloodvessels, particularly those that supply blood to an organ such as thebrain, heart, kidney, liver, lungs, intestines, bladder, stomach,ovaries, and testes, as well as to the periphery, such as the arms andlegs. Preferred blood vessels for analysis include the carotid arteries,coronary arteries, and the aorta. While the instant methods can be usedto detect and analyze vascular plaque in a variety of animal, themethods will most frequently be used on humans.

Typically, the instant methods comprise computationally processingprocessable data from at least one cross section (or portion thereof) ofat least one blood vessel of a patient's vasculature derived from amedical imaging system to determine if the blood vessel (or at least thepart under analysis) comprises at least one plaque component or tissuecorrelated with the presence of plaque. Performance of such methods thusallows assessment of one or measures related to atherosclerosis in atleast a portion of the patient's vasculature.

In preferred embodiments, these methods allow a determination of whethera blood vessel contains plaque, particularly plaque vulnerable torupture. For a particular cross section, the data analyzed may comprisesome or all of the data initially collected. The medical imaging systemused to obtain the initial data may be an invasive or non-invasiveimaging system. Preferred non-invasive imaging system comprises one ormore MRI, CT, PET, thermography, or ultrasound instruments. Instrumentsthat include multiple non-invasive imaging functionalities can also beemployed. Preferred invasive instruments include catheters equipped withone or more sensors. Examples include catheters for intravenousultrasound, angioscopy, intravascular MR, and thermography. Data frominvasive and non-invasive imaging techniques can also be combined foranalysis. Similarly, other or additional data may also be included, forexample, data obtained from the use of contrast agents, labelingmoieties specific for one or more tissues, cell types, or ligands that,for example, comprise tissues or components of healthy or diseasedvasculature, including plaque or components thereof.

MRI-based methods represent a preferred set of embodiments. In suchembodiments, an MRI instrument is used to generate raw magneticresonance data from which processable magnetic resonance data arederived. One or more different imaging modalities, implemented by one ormore different radio frequency pulse sequence series, can allowdifferent tissues and tissue components to be distinguished uponsubsequent analysis. Preferred data types generated by such modalitiesinclude T1-weighted data, T2-weighted data, PDW-weighted data, andTOF-weighted data. Data generated by combinations of one or more ofthese and other data types may also be combined.

While performing the methods of the invention, it may be desirable topre-process and/or normalize data. In any event, the processable dataare computationally processed to determine whether the blood vessel, inthe region of the cross section(s) (or portion(s) thereof) compriseartery and plaque tissue or components thereof. In preferredembodiments, tissue or component type determination is accomplished bycomparing by computer different tissue types identified in the data toone or more of statistical classifiers. Such classifiers can bedeveloped using known outcome data (e.g., by post-operative histologicalexamination, direct tissue inspection, or labeling by one or moreexperts) by any suitable process, including logistic regression,decision trees, non-parametric regression, Fisher discriminant analysis,Bayesian network modeling, and a fuzzy logic system. Components andtissues preferably screened for include muscle, adventitia, calciumdeposits, cholesterol deposits, lipids, fibrous plaque, collagen, andthrombus.

In preferred embodiments, especially those where data from multipleimaging modalities or imaging instruments is used, the data is convertedto a common format. It is also preferably computationally brought intoregistration, often using a landmark, be it one that represents aphysical feature (e.g., an arterial branch point such as the carotidbifurcation) or a computational feature, such as a vessel lumen centroidcalculated from the data being processed. In some embodiments, athree-dimensional model of the blood vessel over at least a portion ofthe region bounded by the most distantly spaced cross sections beinganalyzed can be rendered computationally. A plurality of other analysesor operations may also be performed, including calculation of totalplaque volume or burden, the location and/or composition of plaque, etc.Depending on the analyses or operations performed, the results of theanalysis may be output into one or more output files and/or betransmitted or transferred to a different location in the system forstorage. Alternatively, the data may be transmitted to a differentlocation.

Yet another aspect of the invention concerns assessing effectiveness ofa therapeutic regimen or determining a therapeutic regimen. Such methodsemploy the plaque detection and analysis aspect of the invention, inconjunction with delivering or determining a therapeutic regimen, as thecase may be, depending on the results of the plaque detection, andpreferably classification, analysis. In some embodiments, thetherapeutic regimen comprises administration of a drug expected tostabilize or reduce the plaque burden in a patient over time. Ifdesired, the effect of the therapeutic regimen can be assessed by afollow-up analysis, preferably by performing an additional plaquedetection, and preferably classification, analysis according to theinvention. As will be appreciated, the instant method will be useful notonly in delivering approved treatment strategies, but also in developingnew strategies. As an example, these methods can be used in assessingclinical efficacy of investigational treatments, including those relatedto drugs being assessed for treating cardiovascular and/orcerebrovascular disease.

Another aspect of the invention relates to computer program productsthat comprise a computer usable medium having computer readable programcode embodied therein, wherein the computer readable program code isconfigured to implement an automated method according to the inventionon a computer adapted to execute the computer readable program code.

Computational systems configured to execute such computer readableprogram code represent an additional aspect of the invention, as dobusiness models for implementing such methods, for example, ASP and APIbusiness models. For example, in an ASP model, the medical imagingsystem and computer system configured to execute the computer readableprogram code of the invention are located at different locations.Frequently, the computer system resides in a computational centerphysically removed from each of a plurality of imaging centers, each ofwhich comprises a medical imaging system capable of generating raw datafrom which processable data can be derived. In preferred embodiments, atleast one of the imaging centers communicates raw data to thecomputational center via a telecommunications link.

With regard to computer systems, they typically comprise a computeradapted to execute the computer readable program code of the invention,a data storage system in communication with the computer, and optionallyoperably connected to the computer a communications interface forreceiving data to be processed by, or for sending data after processingby, the computer.

BRIEF DESCRIPTION OF THE FIGURES

These and other aspects and embodiments of the present invention willbecome evident upon reference to the following detailed description andattached drawings that represent certain preferred embodiments of theinvention, which drawings can be summarized as follows:

FIG. 1 is a flowchart that shows an overview of several preferredembodiments of the invention.

FIG. 2 has two panels, A and B. Panel A is an image of generated fromraw magnetic resonance data (in DICOM format) obtained from a commercialMRI instrument that shows the illumination gradients from surface coils.Panel B represents the same image as shown in Panel A after histogramequalization.

FIG. 3 has four panels, A-D. Panel A shows an MRI image derived fromdata obtained using a T1-weighted (T1W) modality. Panel B shows an MRIimage derived from data obtained using a T2-weighted (T2W) modality.Panel C shows an MRI image derived from data obtained using aPD-weighted (PDW) modality. Panel D shows the results of multimodalregistration of the in vivo T1W, T2W, and PDW images.

FIG. 4 is a flowchart showing a process for predictive models useful inthe context of the invention.

FIG. 5 has four panels, A-D, illustrating the process of data labelingfrom MRI images. As will be appreciated, image data, including MRIimages, can be generated from data collected using different protocols(modalities). In this figure, Panel A shows MRI images of a crosssection of a human artery imaged using three standard MRI imagingmodalities: proton density weighted (PDW), T1 relaxation time (T1)weighted (T1W), and T2 relaxation time (T2) weighted (T2W). For easyvisual interpretation, these PDW, T1W, and T2W images (510, 520, and530, respectively) can be combined to create a false-color composite MRimage 540 (Green=PDW, Red=T1, Blue=T2), shown in Panel B. In thecomposite image shown in Panel B, multi-contrast normalized grayscaleimages 510, 520, and 530 were linearly mapped as green, red, and bluechannels, respectively, where black was mapped to zero and white wasmapped to 255 in each color channel to create a color composite imageand render it three-dimensionally using MATLAB. Tissues with similarchemical and environmental properties tend to have similar colors.Additional cues as to tissue type include anatomical location (e.g.,inside or outside the muscle wall, i.e., inside or outside the bloodvessel) and texture (e.g., muscle tends to be striated, whereas softplaque typically appears “mottled”). Expert radiologists can oftenclassify fibrous or vulnerable plaque by detailed manual inspection ofsuch data, but such efforts are extremely time consuming and subjective.To develop an automated system for classifying plaque, the model must be“trained” on known examples (“ground truth”). One can train a model tomimic the performance of an expert, but it is preferred to label theseimages, or the data used to generate images, with the most objectivecriteria possible, such as validation using histopathology sections ofthe tissue. Panel C shows the histopathology (ground truth) of theartery cross section used to generate the images shown in Panels A andB. Panel D of FIG. 5 shows a labeled image used for model training, witheach tissue class of interest labeled with a different target color.Arterial muscle (media, 565) is pink; adventitia (fascia or collagen,570) is bright yellow; thrombus (clotted blood, 575) is red; fibrousplaque (580) is pale yellow; lipid (585) is white, and the vessel lumen(590) is black.

FIG. 6 has three panels, A-C, and presents another example of datalabeling. Panel A shows a false color composite MR image (610) of across section of two arteries. MR image 610 was generated by combininggrayscale MR images generated using three MRI modalities, PDW, T1W, andT2W, as described in connection with the false color image shown in FIG.5. Panel B shows the histopathology of the artery cross sections. PanelC shows the labeled image (630), labeled analogously to the MR image inPanel D of FIG. 5.

FIG. 7 has three panels, A-C, and shows images processed using a K-meansclustering algorithm.

FIG. 8 has two panels, A and B, illustrating the performance of apreferred embodiment as measured against labeled ground truth (leftportion of each panel).

FIG. 9 contains a table (Table A) and three graphs summarizing theperformance of three predictive models for detecting vascular plaque, acomponent thereof (i.e., lipid), and muscle tissue. Table A shows theperformance of the RIPNet models based on the maximum Kolmogorov-Smirnovstatistic (Max-KS) and the Gini coefficient measurements of the ROCcurves shown elsewhere in the figure.

FIG. 10 has three panels, A-C, showing performance of a preferredembodiment of the invention on a low-quality image held out of the modeldevelopment process.

FIG. 11 has two panels, A and B, showing a lumen-centered transformationof the image in Panel A into polar coordinates in Panel B. Thistransform was used to improve the performance of the gross boundarydetection algorithm.

FIG. 12 has two panels, A and B, showing the results of tissuesegmentation algorithm performance on two arterial cross sections. Oncethe tissue segmentation was performed, pixels spuriously labeled asplaque components outside the vessel wall were eliminated, reducingfalse positives. In addition, plaque burden estimates can be obtained bycomparing the ratio of pixels classified as plaque versus the number ofpixels within the wall. In these examples, plaque burdens are estimatedto be 28% and 62%, respectively.

FIG. 13 shows the three-dimensional of part of carotid artery, in theregion of the carotid bifurcation. In the model, the interior boundaryof the arterial wall (1920) and hard plaque (1930) within the vessellumen (1940) is shown, while the exterior boundary of the artery is notshown. Lipid (1910) between the interior surface (1920) and exteriorsurface of the artery wall (not shown) is shown in red. The hard plaquein the model is colored beige.

As those in the art will appreciate, the embodiments represented in theattached drawings are representative only and do not depict the actualscope of the invention.

DETAILED DESCRIPTION

Before the present invention is described in detail, it is understoodthat the invention is not limited to the particular imaging techniques,methodology, and systems described, as these may vary. It is also to beunderstood that the terminology used herein is for the purpose ofdescribing particular embodiments only, and is not intended to limit thescope of the invention described herein.

The present invention concerns automated, objective methods and systemsto detect and analyze plaque in one or more regions of a patient'svasculature. In general, the inventive methods involve a comparison ofdata derived from one, two, or three-dimensional images obtained using amedical imaging system (or data collection precursors to such systems)to examine a patient against a database containing information thatallows the patient-derived data to be classified and plaque detected, ifpresent. Further comparisons allow plaque to be analyzed, for example,classified (e.g., as stable or vulnerable plaque), if desired. Patternrecognition techniques are used to perform these comparisons. Thisinformation, alone or in conjunction with other data about the patient,can be used for various purposes, for example, to determine a course oftherapy, stratify a patient's risk for suffering a subsequent adverseevent (e.g., a stroke or heart attack). Imaging technologies useful inpracticing the invention are those that can be used to generatethree-dimensional images of blood vessels, and include CAT, PET, MRI,and ultrasound. At present, MRI is preferred.

In practice, data for a patient is obtained by sending the patient to anMRI (or other imaging) center that will put the patient into an imagingdevice that generates the basic input data needed to perform thesubsequent analysis. To implement the invention, no additional hardwarewould be needed at imaging centers. Once the raw data are collected, inpreferred embodiments they are sent (e.g., via the Internet as one ormore encrypted electronic data files) to a center for analysis. The dataare then automatically processed to form an individualized product bycomparing the patient's data patterns to a database using a set of oneor more statistical classifiers. An individualized patient product canthen be prepared and sent to the requesting physician. In preferredembodiments, the patient product provides a 3-D visualization of thevasculature of the patient's heart, for example, which may, forinstance, indicate the locations of both total plaque and the subset ofplaque vulnerable to rupture. It may also be useful to quantify thevolume of individual plaques, total plaque, individual vulnerableplaques, and total vulnerable plaque. When used over time to produce aplurality of analyses for a given patient, particularly one undergoingtreatment for an atherosclerotic disease, the methods and systems of theinvention can be used to assess the efficacy of the treatment. Forexample, has the treatment lessened the patient's overall plaque burden(and/or reduced the rate of progression (or expected progression) ofthis burden); has the percentage or amount of vulnerable plaque beenreduced; has the composition of particular plaques changed over time(e.g., become more or less stable, etc.); etc.?

The methods of the invention can readily be embodied in software,hardware, or a combination of these in order to provide automated,non-invasive, and objective detection and analysis (e.g., plaqueidentification and classification) of atherosclerotic (AT) lesions in auser-friendly and reproducible manner. The invention allows researchers,physicians, and patients to readily derive increased benefit fromexisting disease management and/or treatment strategies. These importantdiagnostic and prognostic methods and systems will thus improve therapyand outcomes with respect to the class of diseases that constitute thesingle leading cause of morbidity and mortality in the developed world.

1. Automated Methods for Vascular Plaque Detection and Analysis.

In general, the methods of the invention are based on the computationalanalysis of data for a patient obtained using a medical imaging systemto determine whether a patient suffers from atherosclerosis in at leasta portion of his/her vasculature. To detect vascular plaque, a computerprocesses and compares data using statistical classifiers to determineif one or more regions of the blood vessel(s) under analysis contain atleast one tissue correlated with (i.e., known to be associated with) thepresence of vascular plaque. If desired, plaque, if present, can also beclassified, for example as stable or vulnerable plaque, depending on thetissues identified in the region of the plaque. In addition, assessmentssuch as plaque volume, plaque burden, disease progression, treatmentefficacy, etc. can also be performed.

Initially, raw image data of least one point, line, plane,cross-section, or three- (or more) dimensional image of a patient'sbody, particularly all or a portion of a blood vessel, is gathered usinga medical imaging system. As used herein, “cross-section” will beunderstood to mean that the actual data embodied therein may refer to alesser or greater quantity of data. Preferred medical imaging systemsare non-invasive systems, and include MRI instruments. Raw datacollected from the imaging instrument is then converted into a formsuitable for computer analysis. The analysis is performed using acomputer to compare the processed data for a given cross-section with atleast one, and preferably several, statistically derived classifiers orpredictive models for at least one, and preferably several different,healthy and diseased tissues known to exist in the vasculature. In thisway, a model of at least one cross-section of at least one blood vesselcan be assembled. When data for several or many cross-sections areobtained, a larger model can be assembled that spans the region definedby the various cross-sections. If desired, the resulting model can beused to reconstruct a three-dimensional model of the region(s) of theblood vessel being analyzed, which model can depict various features ofthe blood vessel. For example, the three-dimensional model may show theposition(s) of plaque inside the vessel. Such models can also be used tocalculate a degree of stenosis in one or more regions of a blood vessel,as well as the volume of plaque inside the particular region of thevessel. Plaque volume can be calculated using any suitable approach. Forexample, the total volume of the blood vessel's lumen in the absence ofthe plaque could be calculated, as can the volume of the lumen in thatregion in the presence of the plaque. The difference can be used torepresent the estimated volume of plaque in that region, and the degree(e.g., percentage) of stenosis can also be readily calculated.Similarly, plaque burden can be determined, as can other clinicalmeasures of disease.

A. Representative System Configuration.

Using MRI analysis as a representative example, the overall design of apreferred embodiment of a system for plaque detection and analysisaccording to the invention is schematically illustrated in FIG. 1. Aswill be appreciated, various components of the system are preferablymodular, so that one or more components can be updated or revisedwithout the need for updating or revising the entire system. Inaddition, many of the steps shown are optional, and have been includedin order to describe the currently preferred embodiments of the methodsand systems of the invention. Removal of one or more of these optionalelements, steps, or processes may be desired in a given application.

As shown in FIG. 1, the process begins with patient MRI data beingcollected at an MRI center or other facility (110). The raw data (105)collected are passed on to the plaque detection and analysis systemeither as part of the system resident at the facility where the datawere gathered or at a different facility. For rapid data processing atanother facility, the data are preferably communicated electronically,for example, as an encrypted data file transmitted over the Internet toa facility containing one or more computers configured to process thedata to detect and, if desired, analyze, vascular plaque. Preferably,the raw image data are tested to ensure that it meets minimum qualitystandards (data quality analysis 120), for example, by calculating aPopulation Stability Index. If the data are not of sufficient quality(and can not be rendered to sufficient quality in the particularimplementation of the invention) to render the output reliable, they arenot processed further and a message is preferably transmitted to theimaging center to notice the rejection of the raw image data foranalysis. If desired, another copy of the initial raw data can bere-transmitted, or, alternatively, another set of raw data (105) can becollected and re-submitted for analysis.

After satisfying quality assurance parameters, the raw data are approvedfor further processing. In preferred embodiments, the raw data arepre-processed and/or normalized (step 130) and then computationallyanalyzed to preliminarily identify gross structures in the blood vessel(140). When two or more modes of data are available for analysis,sections and different data modes are then brought into registration(150) using any suitable algorithm configured for computer-basedimplementation. Image transformation, texture processing, and variablecalculation (i.e., image processing, 160) may then be then performed,after which the data can be classified using statistical classifiers orpredictive models to assign tissue classification (170). Gross structureboundaries in the blood vessel can then be determined (step 180), and athree-dimensional reconstruction of the vessel is assembled from thevarious data (185). Thereafter, lesion (here, vascular plaque)diagnostics are performed, after which a three-dimensional model of theblood vessel can be generated, if desired, along with adiagnostic/prognostic report and/or labeled images (195). If desired,the results are then forwarded to the designated recipient, for example,a physician, clinic, or data storage system for subsequent retrieval.

Several steps of the system described above and illustrated in FIG. 1are described in greater detail below.

i. Data Input.

Raw image data (105) for a patient can be presented to the plaqueassessment system according to the invention through any suitablemethod. One such preferred method is an ASP (Application ServiceProvider) model, wherein a patient's raw image data (105) is transmittedfrom an imaging facility via secure Internet connection. Another modelis the (Application Program Interface, or “API”) model, wherein theplaque assessment system is embedded within a software package that isinstalled on-site at the imaging facility.

ii. Image Processing and Formatting.

In preferred embodiments, raw image data are subjected to a qualityassurance examination to ensure it satisfies minimum criteria for dataquality. Data meeting these standards is then pre-processed. Forinstance, data received from different MRI imaging facilities may be indifferent formats, due to the use of different MRI instruments,different versions instrument control software, etc. A preferred commonformat for MRI-derived data are DICOM, although other formats may beadapted for use in accordance with this invention. Also, because ofhardware differences between various MRI instruments and RF coils,resolution and scale can, if desired, be compensated for in a mannerthat produces data that are relatively free of noise and distortion.

As is known, MRI signal intensity drops with distance from the surfaceRF coils (1/R²). As a result, images developed from raw MRI data exhibitan “illumination gradient”, as shown in FIG. 2. If desired, anynow-known or later-developed algorithm useful in correcting for thiseffect can be employed. Suitable methods include histogram equalization(Gonzalas and Woods, Digital Image Processing, 1992 Addison Wesley), aswell as using wavelets to model RF coil function. Other algorithms thatcan be used to correct this effect are available as part of commercialimage processing software tools such as MATLAB (Mathworks, Inc., Natick,Mass.). See also Han, et al. (2001), J. Mag. Res. Imaging, vol.13:428-436.

Robust image discrimination rarely depends on absolute (rather thanrelative) pixel intensity, primarily because intensity often depends onthe particular conditions and imaging machine used to collect the data.Consequently, it is usually valuable to normalize the data to theirhighest pixel intensity in each respective image. Typically, data fromeach data collection modality (e.g., T1, T2, PDW, TOF, etc.) isnormalized independently so that the data for each modality has the samedynamic range. However, additional variables, comparing absoluteintramodal intensity differences, may be created using non-normalizeddata. Multimodal variables, such as the ratio of T1 and T2, for example,measure the ratio of normalized quantities.

The dynamic range of pixel intensities has been observed to contract insome instances, for example, with some in vivo carotid images. By theend of the sequence (closest to the head), the observed resolution canbe quite poor, probably as a result of using a localized neck coil.However, depending on application, for example, estimation of overallplaque burden as opposed to plaque classification or the identificationof microstructures (such as neovascularization or fibrous capthickness), low resolution images may still be useful. In addition,blood suppression pulse sequences can also enhance resolution (Yang, etal. (2003), International J. of Cardiovascular Imaging, vol.19:419-428), as can collection of data using several modalities. Forexample, the fibrous cap of vascular plaque can be distinguished well inTOF images.

iii. Preliminary Gross Structure Identification.

In order to detect and analyze vascular plaque in an imaged crosssection of a patient's body, it is often desirable to identify the bloodvessel(s) sought to be analyzed. Gross tissue identification allows aregion of interest, e.g., a blood vessel, to be extracted for analysisfrom an MRI slice. This can be readily accomplished using morphologicaltechniques to identify the lumen of the vessel, for example. Of course,identification of other gross morphological features, e.g., arterialmuscle, adventitia, etc. can also be employed, alone or in conjunctionwith lumen detection. When lumen detection is employed, once the initiallumen location has been determined, succeeding image slices can use theestimate of the position of the lumen in the preceding slice to make aninitial estimate of the location of the lumen. Once detected, the centerof the lumen (i.e., the centroid of the lumen) is preferablyre-estimated iteratively for each slice. To avoid the compounding ofcentroid estimation errors in successive slices, particularly in thecontext of diseased tissue having irregular features, additionalheuristic algorithms, such as re-registration at a more distal axialposition and interpolation between slices, can be employed.

iv. Image Registration.

The time intervals required to conduct multimodal MRI scans mayintroduce inter- and intra-modal alignment and registration errors, dueto patient motion, heartbeat, breathing, arterial dilation, etc. Forcarotid imaging, a plurality of image slices, for example, 12-20 arepreferably taken in parallel per scan, which takes 3-4 minutes incurrent conventional, commercial MRI instruments. Additional scans arerequired for multimodal images. Hence, the entire process may (atpresent) take 3-20 or more minutes for carotid imaging usingconventional MRI instruments. While gating to heartbeat or respiratorycycle does not yield much benefit on carotid imaging, longer scan times(for example, as may be required for conducting scans using multiplemodalities) may increase the potential for a patient to move during thescanning procedure. For coronary artery imaging, because of the motionof a beating heart, gating may be based on EKG to collect the rawmagnetic resonance data, although doing so often significantly slows theprocess per modality, with times of about 10 minutes/modality not beinguncommon.

When multiple modalities are employed, inter-modal registration oralignment will most likely be required. Reasonable registration can beattained using a straightforward alignment of the lumen centroids.However, due to the high contrast of blood in all MRI modalities, it israther trivial to create a “lumen detector” to center images on animportant reference point, or landmark. Detecting the lumen allowslocation of the gross lumen boundary, which can then be used as thestarting reference point for image registration. Woods, et al. (1998),Journal of Computer Assisted Tomography, vol. 22:139-152. Whilesatisfactory alignment can be achieved through rigid body translationand rotation (see FIG. 3), other more complex methods that considertissue deformation due, for example, to changes in blood pressure, canalso be employed. See, e.g., Dhawan, A. (2003), Medical Image Analysis,IEEE Press Series in Biomedical Engineering.

As will be appreciated, methods that involve more refined alignment,e.g., pixel alignment, preferably employ a metric by which the qualityof the registration can be quantified. Such metrics can be as a simpleas normalized cross-correlation, or they can be more complex, such asthe maximization of mutual information. Viola and Wells (1995),Alignment by Maximization of Mutual Information, InternationalConference on Computer Vision; Wells, et al. (1996), Med Image Anal.,vol. 1(1):35-51. It is important to note that when aligning images ordata sets developed using different imaging modalities, the referenceimage and the image to be aligned frequently display differingcharacteristics. As such, alignment maximization criteria may notexhibit as clear a peak as would be expected if the two images or datasets were collected using the same modality.

For vertical registration, the lumen of the vessel subject to analysisis preferably used align slices from different modalities near a commonanatomical reference point, a computerized fiction (e.g., a lumencentroid), or other landmark. Subsequent slices can then readily bealigned from this common point. For example, a convenient referencepoint in carotid imaging is the carotid bifurcation. Indeed, theanalysis described in the examples below used the carotid bifurcation asan axial reference point. Inter-slice intervals in different modalitiesmay also require linear interpolation algorithms.

v. Image Processing.

In preferred embodiments, processable data (i.e., data configured formanipulation by a computer) are passed through image processingalgorithms to remove noise as well as to synthesize textural featuresand other variables of interest. Although non-parametric regressionmodels (e.g., neural networks or Radial Basis Functions) may be used toestimate any arbitrary, non-linear discriminant function. Cybenko, G.(1989), Mathematical Conti. Signal & Systems, vol. 2:303-314; Hornik, etal. (1989), Neural Networks, vol. 2:359-366; Jang and Sun (1993), IEEETrans. Neural Networks, vol. 4:156-159. As a practical matter, it isuseful to incorporate any known relationships into the variable set, tosimplify the optimization problem. Common techniques include variablelinearization and transforming or combining variables to capturenon-linear relationships, and so on. For example, in building a model todiscriminate seismic signals, it is overwhelmingly more effective tofirst transform the time series into the frequency domain. Dowla, et al.(1990), Bull. Seismo. Soc. Amer., vol. 80(5): 1346-1373. The overallobjective of image processing, then, is to create transformations of theinput image. Types of image processing operations employed fall looselyinto several (not mutually-exclusive) classes, based on theirmathematical objective: noise reduction; dimension reduction; texture orfeature detection; and derived variables (often designed using expertdomain knowledge, although they can be defined usingmathematical/statistical techniques). Examples of variables andtransforms demonstrated to enhance performance of a plaqueclassification system according to the invention are described in theexamples below; however, other image processing techniques known in theart may also be adapted for use in practicing the invention.

vi. Tissue Classification.

After image processing (160), the transformed data are fed intostatistical classifiers to classify each pixel in the image as belongingto one of several tissues, including vascular plaque components.Labeling images is a straightforward process of performing amathematical function on each pixel in the image. One approach for thedevelopment of predictive models is described in Example 2, below. Adetailed example of building predictive models for plaque classificationfrom MRI images is then provided in Example 3.

vii. Tissue Segmentation.

Image segmentation is performed on the output of the tissue classifierin order to highlight tissues of interest, degree of stenosis, etc. aswell as to suppress non-relevant features. In many cases, thedistinction between plaque components and non-pathological tissues isimpossible outside of anatomical context. For example, hard plaque isessentially scar tissue and composed primarily of collagen, as isarterial fascia. Collagen outside the arterial wall is structural, andcertainly not pathological. Likewise, lipid or calcium deposits outsidethe vessel are of no clinical significance in the context of detectingand analyzing plaque inside of blood vessels. Any suitable approach canbe used for this process. In a preferred embodiment, domain knowledgecan be exploited, as some variables lose sensitivity as a function ofradial distance from the lumen boundaries. In another preferredembodiment, excellent results can be achieved using a two-stageapproach, whereby tissue type predictions are passed through a second,gross structure processing module. Essentially, the output of thepredictive models is fed into image processing algorithms (e.g.,gradient-flow and active contour control (Han, et al. (2003), IEEETrans. Biomed. Eng., vol. 50(6):705-710) to define the boundary of thearterial muscle. All pixels outside this boundary may then removed fromconsideration as plaque components or other tissues within arterial wallbounding the interior of the blood vessel. An approach using “activecontour” algorithm or a “snakes” algorithm (Xu, P. (1997), GradientVector Flow: A New External Force for Snakes, IEEE Conference onComputer Visual Pattern Recognition; Xu, P. (1997), Snakes Shapes andGradient Vector Flow, IEEE Transactions on Image Processing) isillustrated on the ex vivo data shown in FIG. 12.

Other segmentation algorithms known to those skilled in the art may alsobe adapted for use in the context of the invention. For example, tissuesegmentation can be accomplished using rules-based methods. The resultsshown in FIGS. 11 and 13 were obtained using such an approach. Suchmethods can also be used in connection with boundary detection methodsthat involve searches for minimum-cost paths (Bishop, C. (1995), NeuralNetworks and Statistical Pattern Recognition, Oxford University Press).In the process used to generate the results shown in FIG. 11, arules-based method was used to transform vessel images, centered on thelumen, to a radial coordinate system that linearized features that areinherently radial.

viii. Three Dimensional Reconstruction.

Once all slices of an MRI scan have been labeled, a full,three-dimensional model of the artery and plaque can be produced, ifdesired. Algorithms that detect gross structure (e.g., lumen andexterior arterial wall) directly from DICOM format data obtained from acommercial MRI instrument (for example, an MRI instrument manufacturedby General Electric) can be used for this purpose. Example 4 describes arepresentative example of how such models can be generated.

ix. Lesion Diagnostics.

Lesion diagnostics, including overall size and degree of stenosis, lipidcontent, plaque size and volume, thrombus, calcification, and so forth,can be estimated from three-dimensional reconstructions of the bloodvessel (Voxels). Of course, imaging modalities that selectively detect aplaque component (for example, lipid) can be employed to generate usefulmodels from less data, in that fewer imaging modalities (e.g., T1, T2,PDW, TOF, etc. in the context of MRI analysis) may be required togenerate models from which vascular plaque can be detected and analyzed(e.g., classified in terms vulnerability to rupture, etc.).

x. Data Output.

The output of the system can be presented in standardized as well ascustom formats to contain such information as may be requested or neededto review the results generated. In some embodiments, the output willconsist of the original data, the data labeled by the predictive models,a three-dimensional model, and a diagnostic report, including riskfactors and recommended therapies, if indicated. Preferably, the outputwill be made available directly to the system, particularly in systemsbased on an API model. In the context of an ASP model, the computersystem that performs the analysis will transmit the output file,automatically or upon receipt of an appropriate command, to a specifiedaddress. Such an address may be an address for an e-mail account of anattending physician, radiologist, and/or specialist, the patient underexamination, the medical imaging facility from which the patient's datawere initially transmitted, etc.

xi. Generalization and Standardization.

As will be appreciated, the automated nature of the methods of theinvention will allow for the development of standardized data analysisprocedures, formats, etc. Also, much of the subjective nature, and thusvariability, of current human expert-based examination of imaging datacan be done away with by implementing the methods and systems of theinvention.

xii. Other Considerations.

As described herein, MRI can be been used to identify morphologicalplaque features, such as plaque size and fibrous cap thickness, withhigh sensitivity and specificity. Furthermore, MRI can discriminateplaque components (e.g., fibrous cap, calcification, lipid content,hemorrhage, etc.) characteristic of vulnerable and stable plaque in allof the major arteries: carotid; coronary; and the aorta. Improvements inimaging protocols have been developed to minimize motion artifacts.Worthley, et al. (2001), Int'l J. Cardiovascular Imaging, vol.17:195-201; Kerwin, et al. (2002), Magnetic Res. In Med., vol.47:1211-1217.

An advantage of MRI is that structures can be imaged using severaldifferent modalities. T1-, T2-, PD-, and TOF-weighted images (T1W, T2W,PDW, and TOFW, respectively) of the same anatomical tissue can be quitedifferent, depending on the chemical components and structure of thetissue. For example, calcification, fibrous tissue, and intra-plaquehemorrhages can be distinguished using T2-weighted images. Calcium isvery hypointense in Proton Density Weighted (PDW) images, while smoothmuscle can be characterized well by a relatively short T. Time-Of-Flight(TOF) weighted images yield good discrimination of intra-plaquehemorrhage and lipid-rich, necrotic cores. Contrast agents can be usedto improve the detection of neovasculature, another indicator of plaquevulnerability. Further, other agents, such as labeled antibodies,vesicles containing targeting moieties specific for a component ofplaque, can also be used to enhance or add to data collected from amedical imaging system for analysis according to the invention.

The inventors have determined that, at present, plaque detection andanalysis according to the instant automated methods based on MRI imagingpreferably uses data derived from two, three, or four different imagingmodes (e.g., T1, T2, PDW, and TOF) or their derivatives (e.g., T1/T2ratios) in order to discriminate plaque components from other tissue ofa blood vessel, although single and other multi-modal analyses are alsowithin the scope of the invention. Integration of information obtainedfrom multiple contrasts would facilitate even more rapid, accurate, andreproducible assessments of plaque presence, location, and composition.Such analyses can then be used to reduce the number of modalitiesnecessary to measure and classify plaque and possibly lead to design ofRF sequences with higher discriminatory power. Similarly, the use ofdata collection modes specific for particular components of vascularplaque will decrease initial data collection times, as will improvementsin imaging equipment hardware, operating software, etc.

2. Applications.

Acute thrombus formation on disrupted/eroded human atheroscleroticlesions plays a critical role on the onset of acute coronary syndromesand progression of atherosclerosis. Pathological evidence has clearlyestablished that it is plaque composition rather than stenotic severitythat modulates plaque vulnerability and thrombogenicity. As will beappreciated, the instant methods and systems can be deployed forautomated image analysis based on pattern recognition for detecting,measuring, and classifying atherosclerotic plaques in vivo, as well astotal plaque burden and related measures. In preferred embodiments,three-dimensional images are derived using MRI. Automation allows fast,objective (observer-independent) data analysis. Such methods will have avariety of applications, including detecting and, if desired, analyzingvascular plaque. Analysis can include, for example, quantitating plaquevolume, determining plaque location, and/or assessing plaquecomposition. Furthermore, the analysis of vascular plaque can focus onone or more regions in vasculature within and/or leading to one or moreregions or organs (e.g., brain, heart, kidney, etc.) in patients with orwithout known cardiovascular disease (which information can help toguide treatment, including surgical intervention and drug therapy),assessing total plaque burden (for example, in the context of patientscreening, disease management, etc.), and risk assessment andstratification. These methods can also be used as standard, objectivediagnostic and prognostic measures, thereby allowing for comparison ofresults between laboratories, throughout longitudinal studies, etc. toassess surrogate end points in clinical trials of drugs and othertreatments, and across different imaging equipment. In a clinicalsetting, these methods will also greatly reduce the diagnostic costsinvolved in measuring the degree of stenosis and detectingthrombosis-prone plaques and reduce the risks to and burdens on patientswho might otherwise have to be subjected to more invasive diagnosticmethods, while at the same time providing much more useful informationthan can be obtained using existing methods.

A. Cardiovascular Disease.

Thus, one context in which the invention has application concernscardiovascular disease. As is known, cardiovascular disease is thesingle leading cause of death in both men and women. About one-half ofindividuals in developed nations die of cardiovascular disease, and manymore will suffer complications associated with cardiovascular diseaseand the accompanying lower quality of life. In the U.S. alone, over $15billion is spent annually on products that visualize the heart andplaque. Recent findings show that vulnerable, not stable, plaqueruptures to cause heart attacks and strokes. Significantly, about 70% ofplaque that ruptures to produce heart attacks comes from areas of thevasculature where there is little plaque. To date, however, noobjective, rapid method has been developed to distinguish betweenvulnerable, unstable plaque that is likely to rupture and cause athrombosis that can lead to a heart attack or stroke, and stable plaque.The instant invention addresses this significant unmet need by providingnon-invasive, objective, and rapid methods to detect and analyze plaquethroughout the vascular system, particularly in the vasculature of thebrain, neck, and heart.

i. Pre-Operative Lesion Diagnostics and Patient Screening.

All current American Heart Association guidelines are based on degree ofstenosis and symptom status, without reference to plaque composition.Clearly, more precise pre-surgical diagnostics (for example, plaquecomposition, e.g., calcification, lipid content, thrombosis, fibrous capthickness, and so on) will significantly improve the pre-surgical riskestimates, allowing clinicians to more reliably assess the relative riskof surgery over pharmaceutical intervention.

ii. Treatment.

Many cardiovascular and cerebrovascular preventive measures andtreatments are assigned to patients based on an estimation of thepatient's cardiovascular disease (CVD) or cerebrovascular risk. Forpurposes of this description, CVD will be discussed as therepresentative example of atherosclerotic diseases to which theinvention in general relates. Thus, the Joint National Committee'shypertension guidelines, and the Adult Treatment Panel's/NationalCholesterol Education Panel's cholesterol guidelines define eligibilityfor treatment by expected CVD risk: that is, they define treatmentthreshold percentages, or levels of blood pressure or cholesterol atwhich treatment is initiated, based on CVD risk estimates. Additionally,they define goals of treatment (treatment targets) by expected CVD risk:that is, aggressiveness of treatment, or levels of blood pressure orcholesterol down to which treatment should be advanced.

This is theoretically justified because persons at higher CVD risk havemore risk to reduce: the same fractional reduction in risk leads to alarger absolute reduction in risk in those at higher baseline CVD risk,with the greater CVD risk reduction providing greater cost effectivenessof treatment (fewer needed to treat to prevent a CVD event or death);and greater likelihood that the (greater absolute) benefits of treatmentwill exceed treatment harms.

Current approaches to CVD risk estimation (on which treatment thresholdsand targets are predicated) do not incorporate information related tovulnerable plaque. Since vulnerable plaque is a key determinant of CVDrisk (arguably the most important determinant), and since this inventionallows vulnerable plaque to be detected and analyzed in an objective,automated, and accessible, the accuracy of CVD risk predictions (andrisk predictions targeted to different end-organs) can be greatlyimproved, permitting markedly improved targeting of treatments.

The improved CVD risk prediction (and risk stratification) fromeffective assessment of, for example, vulnerable plaque, total plaqueburden, etc., may have important cost-saving and life-savingimplications. Improved targeting of treatments to those truly at riskwill save lives for the same cost, and save money for the same savingsof life.

Plaque detection and characterization will also permit better decisionsregarding who merits medical treatment, and what medical treatments willbest serve a particular patient. This may include allocation of (costly)statin cholesterol-lowering drugs (e.g., atorvastatin, simvastatin,pravastatin, lovastatin, rosuvastatin, and fluvastatin), which currentlyaccount for the greatest expenditures for any prescription medication inthe world, with a $20 billion dollar annual market, and whose usage isexpected rise markedly with aging populations. More generally, plaquedetection and analysis can also improve treatment decisions fortreatment regimens that attack cardiovascular risk through any of asuite of mechanisms, including lowering blood pressure (such as thiazidediuretics, e.g., hydrochlorothiazide, beta blockers such as atenolol,angiotensin converting enzyme inhibitors such as fosinopril, angiotensinreceptor blockers such as irbesartan, calcium channel blockers such asnifedipine (diltiazem and verapamil), alpha blockers such as prazocin(terazocin), and vasodilators such hydralazine), stabilizing plaque (ascan be achieved by some statins), reducing lipids (as can accomplishedusing statins; fibric acid derivatives like gemfibrozil or fenofibrate;niacin or variants like niaspan; bile acid sequestrants like colestipolor cholestyramine; or blockers of cholesterol absorption likeezetimibe), reducing inflammation, and/or serving antiplatelet (e.g.,aspirin, clopidogrel, etc.) or antithrombotic effects (e.g., tissueplasminogen activator or streptokinase), among others. Of course,depending on the particular patient and condition to be treated, it maybe desirable to combine one or more of the foregoing therapies, alone orin combination with other treatments.

The improvements in targeting surgical treatments to those at greatestneed may be even more important, since the potential costs and risksassociated with surgery should be borne only by those for whom the truerisks of the problem exceed the risks of the surgery. Vulnerable plaqueassessment may greatly improve determination of whether a patient trulyhas this level of risk.

iii. Drug Development.

Researchers have used manual evaluation of non-invasive patient imagedata to monitor the efficacy of cholesterol-lowering drugs inlongitudinal studies. However, manual examination is too expensive forgeneral clinical diagnostics. In contrast, the automated methods andsystems of the invention can be used to rapidly generate a statisticallyreliable estimates of plaque composition (e.g., calcification, lipidcontent, thrombosis, fibrous cap thickness, and so on), total plaqueburden, vulnerable plaque burden, the ratio of vulnerable to stableplaque, or lipid deposits, to be used as a surrogates of clinicaloutcomes (e.g., rupture, stroke, MI), greatly reducing the time and costof research. Another major advantage afforded by the invention is tosignificantly reduce the number of patients and the length of follow-uprequired to demonstrate the effectiveness of cardiovascular andcerebrovascular drugs, including those undergoing clinical trials. Forinstance, given the significant clinical benefits associated to the useof statins, it might be unethical to perform any new trials that includea placebo. Therefore, to demonstrate a significant advantage overcurrently used cardiovascular drugs, trials may require at least a fewthousands patients and 3-5 years of follow-up.

iv. Enhanced Diagnostics.

Another application for the instant methods and systems concernsprovision of superior diagnostic and prognostic tools to patients andphysicians. In this regard, plaque detection and analytical data andresults derived from use of this invention can be combined with datafrom other sources to provide even more advanced diagnostic products andservices. For example, the Framingham Heart Study database has been usedextensively to create scorecards to estimate the risk of cardiovasculardisease (CVD). This landmark dataset was developed from tracking 5,209subjects over time, and from whom a host of predictor variables havebeen obtained, including age, gender, measures of cholesterol andhypertension, demographic factors, medications, diabetes status, alcoholconsumption, smoking history, history of cardiac events (e.g.,myocardial infarction (MI), angina pectoris, tachycardia, andbradycardia), revascularization, coronary artery bypass graftprocedures, stroke, levels of analytes in the blood (e.g., creatinine,protein), classification of personality (e.g., type A), and a number of“emerging” risk factors, including levels of C-reactive protein, VCAM,ICAM adhesion molecules, and others in the blood. Outcomes accessedinclude death (cause-specific), and cardiovascular events, including MI,stroke, and sudden death.

There are no MR scans in the Framingham dataset; however, in vivopatient MRI from more recent longitudinal statin drug trials containsboth MRI images and pertinent patient histories and risk factors (e.g.,blood pressure, cholesterol levels, etc.) is available. For example, aresearch team at Mt. Sinai collected MR images at six-month intervalsover two years (Woods, et al. (1998), Journal of Computer AssistedTomography, vol. 22:139-152). By combining the results obtained fromusing the instant methods with one or more other data points correlatedwith CVD, improved even better diagnostic procedures can be implemented.

B. Stroke.

Better knowledge of the composition of atherosclerotic lesions will alsoallow for more accurate patient risk stratification for stroke,facilitating the selection of appropriate therapies. Approximately 25%of strokes are related to occlusive disease of the cervical internalcarotid artery. Treatment options include anti-platelet therapy,endarterectomies, stenting, and angioplasty. Of these treatments,carotid endarterectomy (proactive surgical removal) is the preferredtreatment option for advanced carotid lesions, with over 120,000 ofthese surgeries being performed every year in the United States. Severallarge clinical studies, including the North American Symptomatic CarotidEndarterectomy Trial (NASCET), the Asymptomatic Carotid AtherosclerosisStudy (ACAS), and the European Carotid Surgery Trial group (ECST), haveshown this procedure to significantly reduce the risk of stroke undercertain limitations. For symptomatic patients with 70% stenosis, theoverall reduction in two-year risk of stroke has been estimated to be17%. Surgery also increases the risk of perioperative events; mortalityincreases from 0.3% to 0.6% in surgery patients; major stroke increasesfrom 3.3% to 5.5%; and cerebrovascular events increase from 3.3% to5.5%. For asymptomatic patients with greater than 60% stenosis, theaggregate risk of stroke and perioperative stroke or death is estimatedto be 5.1% for surgical patients, compared to 11% for those treatedmedically. Moreover, evidence suggests that adverse outcome estimatesderived from trials underestimate the likelihood of adverse outcomes inreal world application—further increasing the importance of identifyingthose for whom true benefit is likely.

Use of the instant methods will allow patients to be better assessed sothat the appropriate therapy can be implemented. Also, as withcardiovascular disease, screening will allow patients to be diagnosedmuch earlier in the development of disease, enabling early therapeuticintervention and much greater risk reduction over time.

3. Computer-Based Implementations.

The various techniques, methods, and aspects of the invention describedabove can be implemented in part or in whole using computer-basedsystems and methods. Additionally, computer-based systems and methodscan be used to augment or enhance the functionality described above,increase the speed at which the functions can be performed, and provideadditional features and aspects as a part of or in addition to those ofthe present invention described elsewhere in this document. Variouscomputer-based systems, methods and implementations in accordance withthe above-described technology are now presented.

The various embodiments, aspects, and features of the inventiondescribed above may be implemented using hardware, software, or acombination thereof and may be implemented using a computing systemhaving one or more processors. In fact, in one embodiment, theseelements are implemented using a processor-based system capable ofcarrying out the functionality described with respect thereto. Anexample processor-based system includes one or more processors. Eachprocessor is connected to a communication bus. Various softwareembodiments are described in terms of this example computer system. Theembodiments, features, and functionality of the invention in thisspecification are not dependent on a particular computer system orprocessor architecture or on a particular operating system. In fact,given the instant description, it will be apparent to a person ofordinary skill in the relevant art how to implement the invention usingother computer or processor systems and/or architectures.

The various techniques, methods, and aspects of the invention describedabove can be implemented in part or in whole using computer-basedsystems and methods. Additionally, computer-based systems and methodscan be used to augment or enhance the functionality described above,increase the speed at which the functions can be performed, and provideadditional features and aspects as a part of or in addition to those ofthe present invention described elsewhere in this document. Variouscomputer-based systems, methods and implementations in accordance withthe above-described technology are now presented.

The various embodiments, aspects, and features of the inventiondescribed above may be implemented using hardware, software, or acombination thereof and may be implemented using a computing systemhaving one or more processors. In fact, in one embodiment, theseelements are implemented using a processor-based system capable ofcarrying out the functionality described with respect thereto. Anexample processor-based system includes one or more processors. Eachprocessor is connected to a communication bus. Various softwareembodiments are described in terms of this example computer system. Theembodiments, features, and functionality of the invention in thisspecification are not dependent on a particular computer system orprocessor architecture or on a particular operating system. In fact,given the instant description, it will be apparent to a person ofordinary skill in the relevant art how to implement the invention usingother computer or processor systems and/or architectures.

In general, a processor-based system may include a main memory,preferably random access memory (RAM), and can also include one or moreother secondary memories, including disk drives, tape drives, removablestorage drives (e.g., pluggable or removable memory devices and tapedrives, CD-ROM drives, DVD drives, floppy disk drives, optical diskdrives, etc.). In alternative embodiments, secondary memories includeother data storage devices for allowing computer programs or otherinstructions to be called or otherwise loaded into the computer system.

A computer system of the invention can also include a communicationsinterface (preferably compatible with a telecommunications network) toallow software and data to be transferred to, from, or between thecomputer system and one or more external devices. Examples ofcommunications interfaces include modems, a network interface (such as,for example, an Ethernet card), a communications port, a PCMCIA slot andcard, etc. Software and data transferred via communications interfacewill be in the form of signals that can be electronic, electromagnetic,optical, or other signals capable of being received by thecommunications interface. These signals are usually provided tocommunications interface via a channel that carries signals and can beimplemented using a wireless medium, wire, cable, fiber optics, or othercommunications medium. Some examples of a channel include a phone line,a cellular phone link, an RF link, a network interface, and othercommunications channels.

In this document, the terms “computer program product” and the likegenerally refer to media such as removable storage device, a diskcapable of installation in disk drive, and signals on channel. Thesecomputer program products provide software or program instructions tothe computer processor(s). Computer programs (also called computercontrol logic) are usually stored in a main memory and/or secondarymemory. Computer programs can also be received via a communicationsinterface. Computer programs, when executed, enable the computer systemto perform the features of the present invention as described herein. Inparticular, the computer programs, when executed, enable theprocessor(s) to perform the features of the present invention.Accordingly, computer programs represent controllers of the computersystem.

In embodiments where the invention is implemented using software, thesoftware may be stored in, or transmitted via, a computer programproduct and loaded into computer system using any suitable device orcommunications interface. The control logic (software), when executed bythe processor(s), causes the processor to perform the functions of theinvention as described herein. In other embodiment, the methods of theinvention implemented primarily in hardware, or a combination ofhardware and software, using, for example, hardware components such asPALs, application specific integrated circuits (ASICs), or otherhardware components. Implementation of a hardware state machine so as toperform the functions described herein will be apparent to personsskilled in the relevant art(s).

EXAMPLES

The following Examples are provided to illustrate certain aspects of thepresent invention and to aid those of skill in the art in practicing theinvention. These Examples are in no way to be considered to limit thescope of the invention in any manner.

Example 1 Image Processing

This example describes several particularly preferred techniques forprocessing image data in the context of this invention, including noisereduction, dimension reduction, and texture processing.

A. Noise Reduction.

Composite multi-contrast images were processed in order to reduce noiseand introduce smoothing. In each case, the image was firstmedian-filtered to remove noise with impulse characteristics, thensmoothed with an adaptive Wiener filter that adjusts to statistics inthe surrounding ‘N’ pixel neighborhood. The mean and variance areestimated from the intensities ‘a’ at pixel locations n₁, n₂:μ=(1/N ²)Σa(n ₁ ,n ₂);σ²=(1/N ²)Σa ²(n ₁ ,n ₂)−μ²

These estimates are then used to assign the parameter ‘b’ to a Wienerfilter^(i):

${b\left( {n_{1},n_{2}} \right)} = {\mu + {\frac{\left( {\sigma^{2} + v^{2}} \right)}{\sigma^{2}}\left( {{a\left( {n_{1} + n_{2}} \right)} - \mu^{2}} \right)}}$A 2-dimensional convolution was performed on each image intensity planewith the coefficients b above.

B. Dimension Reduction.

Because of the number of data points and variables to be processed, inorder to minimize the effects of noise, it is preferred to reduce thedimensionality of the dataset to create fewer, but morestatistically-significant, variables. Cluster analysis is only one ofmany methods employed to reduce noise and dimensionality of raw datagenerated by an imaging instrument to its most salient features. K-Meansclustering is one example of a clustering algorithm. In such analgorithm, ‘K’ classes are formed, the members of which reside in(feature-space) locations that are least distant from the estimatedcentroid of each class. This approach makes an initial estimation at thecluster centroids and then re-estimates those centroids according toupdated class memberships. This method makes an initial estimation atthe cluster centroids and then re-estimates those centroids according toupdated class memberships.

The steps underlying the K-means clustering algorithm are:

-   -   i. select a number of clusters ‘k’ with initial centroids;    -   ii. partition data points into k clusters by assigning each data        point to its closest cluster centroid;    -   iii. compute a cluster assignment matrix; and    -   iv. estimate the centroids of each cluster.        Steps ii-iv are repeated until stopping criteria are reached,        typically when the members stop changing cluster membership. See        Bishop (1995), Neural Networks and Statistical Pattern        Recognition, Oxford University Press. Exemplary cluster analysis        results are shown in FIG. 7. From the figure it is clear that        some clusters had high correlation to particular tissue types.        Texture clustering produced results that were visually quite        satisfying, but statistically not as good as a predictive model.        The K-means cluster categories were thus used as inputs into the        predictive models. Other tools known in the art may also be used        to reduce dimensionality, including approaches that combine the        theoretical nonlinear curve fitting capability of the typical        artificial neural network (ANN) with the stability of        hierarchical techniques (Bates White, LLC software, RDMS™). As a        result, the estimation routines are exposed only to those inputs        that are known to have some predictive power on their own, and        that may also embody a number of the most useful underlying        nonlinear effects in the model. These steps allow the ANN        training stage to focus on a problem with lower dimensionality        and with less nonlinearity in the parameters than otherwise        required. Still other techniques include Principal Component        Analysis, Independent Component Analysis (Bell and Sejnowski        (1995), Neural Computation, vol. (7)6:1129-1159), and local        information metrics (Haralick, R. (1979), Proc. IEEE, vol.        67(5)).

C. Texture Measures.

Tissues and plaque components are visually distinguishable by theirtexture. For example, muscle and collagenous tissues are often striated,while necrotic cores appear mottled. There are no formal mathematicaldefinitions of texture, but these features have mathematical correlates,such as information content, spatial frequencies, and so forth. Onecommonly used classification distinguishes 28 texture measures. Here,two classes of texture measures, statistics on the local intensityvariations and spatial frequency, were used. Statistical pixel measuresused were standard statistical quantities, applied to neighborhoods ofvarious sizes. A discrete cosine transform was used to generate anestimate of spatial spectral energy for both ‘x’ and ‘y’ orientations.For example, a pixel area that is rich in fine detail has a greaterproportion of energy in higher spatial frequencies. The expression for a2D DCT is:

${B_{pq} = {\alpha_{p}\alpha_{q}{\sum\limits_{m = 0}^{M - 1}{\sum\limits_{n\; 0\; 1}^{N - 1}{A_{mn}\cos\;\frac{{\pi\left( {{2m} + 1} \right)}p}{2M}\cos\;\frac{{\pi\left( {{2n} + 1} \right)}q}{2N}}}}}},\begin{Bmatrix}{0 \leq p \leq {M - 1}} \\{0 \leq q \leq {N - 1}}\end{Bmatrix}$ ${\alpha_{p} = \begin{Bmatrix}{{1/\sqrt{M\;}},{p = 0}} \\{\sqrt{2/M},{1 \leq p \leq {M - 1}}}\end{Bmatrix}},{\alpha = \begin{Bmatrix}{{1/\sqrt{N}},{q = 0}} \\{\sqrt{2/N},{1 \leq q \leq {N - 1}}}\end{Bmatrix}}$

D. Derived Variables.

Derived variables are synthesized from basic entities purported to havepredictive properties. Two variables falling under this category includeproducts and ratios of raw variables and other combinations of the threedata types. Three types of derived variables found to have strongdiscriminatory power include: (i) a “Fat Detector,” defined as the ratioT1/T2 that is useful in the detection of lipids; (ii) an axes rotation:YCbCr: In this format, luminance information is stored as a singlecomponent (Y), and chrominance information is stored as twocolor-difference components (Cb, Cr). Cb represents the differencebetween the blue component and a reference value. Cr represents thedifference between the red component and a reference value; and (iii) alocal environment variable, based on the geometric distance from thelumen boundary. Examples of other potentially valuable image featuresare given in Table 1, below.

TABLE 1 Examples of image processing variables and transforms ClassTransform Textural Measures Mean, Variance, Skew, Median, Inter-QuartileRange, Minimum, Maximum, Standard deviation, Range, Kurtosis, LocalInformation Content Textural Measures Discrete Cosine Transform, wavelettransforms, orientation Derived Variables Ratios (e.g., T1/T2), distancefrom lumen, distance from arterial wall Derived Variables RGB to YCbCrColor Axes Rotation Dimension Reduction K-Means Clustering, Principalcomponents, Independent Component analysis

Example 2 Predictive Model Development

A “model” is a mathematical or statistical representation of data or asystem, used to explain or predict behaviors under novel conditions.Models can be mechanistic (commonly employed in the physical sciencesand engineering) or empirical/statistical (wherein the model predictionsdo not purport to explain the underlying causal relationships). Tworelevant applications of statistical modeling are to develop statisticalclassifiers and predictive models. Statistical classifiers are designedto discriminate classes of objects based on a set of observations.Predictive models attempt to predict an outcome or forecast a futurevalue from a current observation or series of observations. Thisinvention employs both types of models: statistical classifiers are usedto classify tissue and plaque components; and predictive models are usedto predict risks associated with, for example, cardiovascular disease(CVD).

The process of model development depends on the particular application,but some basic procedures, illustrated schematically in FIG. 4, arecommon to typical model development efforts. First, a modeling datasetmust be constructed, including a series of observations (“patterns”) andknown outcomes, values, or classes corresponding to each observation(referred to as “labeled” or “target” values). In FIG. 4, this ischaracterized as dataset construction 410. This modeling dataset is usedto build (or “train”) a predictive model. The model is then used toclassify novel (or unlabelled) patterns. Model development is often aniterative process of variable creation, selection, model training, andevaluation, as described below.

A. Dataset Construction.

The first step in the model building process is generally to assembleall the available facts, measurements, or other observations that mightbe relevant to the problem at hand into a dataset. Each record in thedataset corresponds to all the available information on a given event.In order to build a predictive model, “target values” should beestablished for at least some records in the dataset. In mathematicalterms, the target values define the dependent variables. In the exampleapplication of CVD risk prediction, targets can be set using observedclinical outcomes data from longitudinal clinical studies. In thecontext of plaque detection and analysis (e.g., classification), thetargets correspond to, as examples, images labeled by a human expert orvalidated by histological examination. FIG. 5 illustrates the datalabeling process used in one such application. In this example, eachpattern/target pair is commonly referred to as an exemplar, or trainingexample, which are used to train, test, or validate the model. As willbe appreciated, what constitutes a pattern exemplar depends on themodeling objective.

i. Data Splitting.

As illustrated in FIG. 4, the implementation of models typicallyincludes data splitting (step 420). Most model development effortsrequire at least two, and preferably at least three data partitions, adevelopment data set (data used to build/train the model) 427, a testdataset (data used to evaluate and select individual variables,preliminary models, and so on) 425, and a validation dataset (data toestimate final performance) 429. To serve this purpose, the initial dataare randomly split into three datasets, which do not necessarily haveequal sizes. For example, the data might be split 50% development (427),25% test (425), and 25% validation (429). The model is initiallydeveloped using development data (427). The resulting performance on thetest data (425) is used to monitor issues such as any over-fittingproblems i.e., the model should exhibit comparable performance on boththe development data (427) and test data (425). If a model has superiorperformance on development data (427) relative to test data (425), themodel is adjusted until the model achieves stable performance.

To verify that the model will perform as expected on any independentdataset, ideally some fraction of the data are set aside solely forfinal model validation. A validation (or “hold-out”) data set 429consists of a set of example patterns that were not used to train themodel. A completed model can then be used to score these unknownpatterns, to estimate how the model might perform in scoring novelpatterns.

Further, some applications may require an additional, “out-of-time”validation set, to verify the stability of model performance over time.Additional “data splitting” is often necessary for more sophisticatedmodeling methods, such as neural networks or genetic algorithms. Forexample, some modeling techniques require an “optimization” data set tomonitor the progress of model optimization.

A further aspect of modeling is variable creation/transformations, asshown in step 430 of FIG. 4. In this processing, the objective isprecision and the incorporation of domain knowledge. Raw data values donot necessarily make the best model variables due to many reasons: datainput errors; non-numeric values; missing values; and outliers, forexample. Before running the modeling logic, variables often need to berecreated or transformed to make the best usage from the informationcollected. To avoid the dependence between development data, test dataand validation data, all the transformation logic will preferably bederived from development data only.

In conjunction with transforming the variables as desired and/or asneeded, the modeling process includes the step 440 of variableselection. Thereafter, the model development may include training of themodel 450 in conjunction with testing of the model. This may then befollowed by model validation.

The results of the model validation 460 reveal whether performanceobjectives 470 were attained. As shown in FIG. 4, if the performanceobjectives have been attained, then the modeling process is terminatedin step 480. Should the performance objectives not be attained, furtherdevelopment of the model may be required. Accordingly, the process ofFIG. 4 may return to step 430 so as to vary the variable creation ortransformations in order to achieve better performance.

Example 3 Plaque Classification

This example describes a preferred embodiment of the invention fordetecting and classifying plaque using statistical classifiers.Initially, effort was directed to building a system using a set ofmodels for detecting three key components of atherosclerotic plaque inMR (magnetic resonance) images of ex vivo blood vessels. The system alsodetected arterial muscle tissue that, when combined with the plaque andlipid detection systems, allowed the full artery to be identified in theimage and plaque burden estimates to be computed. This system is fullyautomated, and in this example, the only human intervention in thedetection and analysis process came during the collection of the rawmagnetic resonance data from the MRI instrument. Using this system, asuccess rate equal or superior to the performance of a human expertradiologist was achieved in plaque component classification.

In this example, predictive models were trained to identify three tissuetypes: plaque, lipid, and muscle. The plaque detector was trained usinga labeling of the example images that identified hard plaques. The lipiddetector was trained on a smaller set of images where lipids could beidentified and labeled. The muscle detector is used to separate arterialwall tissue from other parts of the vessel shown in the images.Additional models may be developed to detect calcified tissue, thrombus,and other non-pathological tissues. With a proper identification of thearterial walls it is possible to compute plaque burden estimates withinthe vessel given the outputs of the other models.

The goal of predictive modeling is to accurately predict theground-truth classifications for each pixel of an image based on thecharacteristics of the MRI image at that pixel and its immediatesurroundings. The predictive modeling began once the image processingsteps were completed and the original images were transformed intocolumnar data representing each pixel as a record. Each pixel recordcontained one variable identifying the ground-truth classification foreach pixel, and over four hundred additional variables capturingcharacteristics derived from the image processing steps. The challengeof the predictive modeling was to sift through these hundreds ofpotential variables, and thousands of permutations of the variables, tocome up with the most predictive combination.

In a preferred embodiment, the artificial neural network (ANN) modelingapproach known as the Relevant Input Processor Network (RIPNet™; BatesWhite, LLC, San Diego, Calif.; Perez-Amaral and White (2003), OxfordBulleting of Economics and Statistics, vol. 65: 821-838); however,standard linear and non-linear regression techniques known to those inthe art (such as linear and logistic regression, decision trees,non-parametric regression (e.g., using neural networks or radial basisfunctions), Bayesian network modeling, Fisher discriminant analysis,fuzzy logic systems, etc.) could also be used. RIPNet™ was developedspecifically to address the problem of how to identify a networkarchitecture with many potential variables, while avoiding overfit. Thetypical problem associated with neural network estimation is that thefunctional form embodied in these models is essentially “too flexible”.Standard ANN approaches specify a level of model flexibility that, ifleft to be estimated automatically (unless a test or cross-validationprocess is also included automatically), summarize not only the signalin the data but the noise as well. This results in overfit, a situationin which the model does not generalize well for information notcontained in the training data. Procedures such as optimal stoppingrules have gained wide acceptance as a method for stopping the networktraining procedure (essentially a least-squares fitting algorithm) at apoint before fitting of noise begins. These procedures deal with thesymptom, but not the cause of model overfit. Model overfit in ANNs isfundamentally caused by an over-complexity in the model specificationthat is directly analogous to the overfit problems that may beencountered with linear models. When one encounters overfit in a linearmodel, one solution is not generally to modify the least squares fittingroutine, but to simplify the model specification by dropping variables.Another solution is to use a cross-validation data set to indicate whento stop fitting.

RIPNet embodies a fundamentally different approach to neural networkestimation that is aimed directly at identifying the level of modelcomplexity that guarantees the best out-of-sample prediction performancewithout ad-hoc modifications to the fitting algorithms themselves. Thereare five major steps to producing models using the RIPNet approach,discussed in greater detail throughout the results sections that follow:(1) dataset creation, labeling, and sampling; (2) anomalous datadetection; (3) variable pre-selection and transform generation; (4)predictive model estimation and variable selection; and (5) final modelvalidation.

i. Dataset Creation.

The models were developed from pooled datasets of MRI images to predictthe presence each of several major tissue features on a pixel-by-pixelbasis. Ten labeled images yielded 112,481 useable tissue observations(i.e., pixels). The results demonstrate a scalable approach to featuredetection that does not rely on the specifics of vessel geometry or theresolution of the image to obtain clinically relevant, reliable results.

The image data set used for model training and estimation consisted often ex vivo arterial sections that represent all of the arterialcross-section images available for this project. Labeling of the plaque,muscle, and lipid components of each artery was performed by directcomparison with histology. All images examined contained significantexamples of hard plaque that was labeled for estimation. Muscle wasclearly identifiable and labeled in most of the images, but in some, forexample, the lower-quality image in FIG. 10, one of the arteriespresents a histological challenge. Only three of the images containexamples of lipid, with the image in FIG. 5 having the largest suchexample.

The data extracted from these images was in the form of pixels that weretreated as separate data points. The target variable for the modelingprocess is an indicator variable, which is one if the pixel belongs tothe target class, and zero otherwise. This indicator is based on thelabeling of the image. Associated with each pixel in the MRI image arethree variables indicating intensity in the T1, T2, and PD modalities.The dynamic range of these intensities is 0 to 255, taking on onlyinteger values (8 bit color depth). These data were heavily processed togenerate a large number of additional variables summarizing such thingsas average intensity in the neighborhood of the pixel, and other moresophisticated transforms such as local texture measures.

Because the specimens were mounted on slides, which do not generateuseful MRI information, a large number of pixels were dropped becausethey did not contain relevant data. Here, these pixels were identifiedas those for which the T1, T2, and PD indicator variables were allsimultaneously zero. This procedure was conservative, and allowed somepixels with random noise into the dataset, although this has no impacton the performance of the algorithms. Over 50% of each original imagewas omitted in this way.

To maximize outcomes, the RIPNet procedure prefers that data be split atmultiple stages in the modeling process so that there are systematictests of real-world performance throughout. For this reason, some of theimages were completely reserved as a test of performance. The datasetsused for modeling were as follows: training, used to estimate modelparameters; validation, used in the cross-validation of modeling resultsto verify performance of selected variables based on out-of-sampleentropy measures and pseudo-R-squared measures; testing, onlyinfrequently used for comparing the relative performance of alternativemodel specifications, this dataset was developed and used by the DataMiner's Reality Check™ algorithm (White, H. (2000), Econometrica, vol.68:1097-1126; U.S. Pat. Nos. 5,893,069 and 6,088,676) because thevalidation dataset was heavily mined; and hold-out, which data (threeimages and over 45,000 observations) was held entirely outside theestimation and validation processes in order to provide real-worldexamples to the model. All records were selected into their respectivesamples at random.

ii. Anomalous Data Detection.

An anomalous data detection algorithm was developed to identify outliersin the data. Here, the anomaly detector was a form of clusteringalgorithm that allows multivariate outliers to be identified among thedata. An anomaly was identified as a record that is distant (as measuredby L1-norm) from its k nearest neighbors. The data were separatedbetween the target=0 sample and the target=1 sample so that anomaliescould be identified relative to these separate groups. In this instance,k=10 was selected. This procedure typically is necessary to identifyrecords that might have unusual leverage on the model estimationroutines. However, if the image raw data from the imaging instrument isrelatively clean, as is the case with most MRI data, no major outliersmay be identified.

Table 2, below, set outs the contents of several anomaly variables

TABLE 2 Anomaly variable contents Plaque Lipid Muscle PD mean 3 pixelradius T1 min. 3 pixel radius T2 min. 3 pixel radius T2 mean 3 pixelradius T2 min. 3 pixel radius T1 min. 5 pixel radius T2 median 3 pixelradius T1 min. 4 pixel radius T2 mean 5 pixel radius T1 mean 5 pixelradius T1 min. 5 pixel radius T2 min. 5 pixel radius k-means variable T2min. 5 pixel radius k-means variable

The anomalous data detection engine also generated an anomaly variable.The anomaly variable translated the two distance measures for eachobservation (relative to the target=0 sample and relative to thetarget=1 sample) into a likelihood ratio statistic. This statisticembeds relative distances to the target samples in a transformation ofinput variables, which is a powerful predictor in some instances. Theanomaly variable for each model is composed of up to five continuousinput variables, as shown in Table 2, above.

iii. Variable Pre-Selection and Transform Generation.

Next, an additional phase of variable transform generation and apreliminary elimination of non-predictive variables to reduce datasetsizes were undertaken. The transforms generated at this stage includedthe following for all of the variables on the input dataset: grouptransforms, which are univariate continuous variables grouped intodecile bins that were then combined through a clustering algorithm toachieve the smallest number bins without significantly reducingpredictive performance; cross-products, which are univariate continuousand discrete variables interacted with one another and grouped intobinned categorical variables using the aforementioned clusteringalgorithm; and beta transforms, which are a flexible functional formbased on fitting beta distribution functions to the data and computinglikelihood ratios.

All of the variables generated up to this point were tested forperformance on the target variable using an out-of-sample pseudoR-squared statistic. A straightforward entropy calculation contrastedthe distributions of the independent variables given the state of thedependent variable, which was then summarized in a pseudo R-squaredstatistic for the validation sample. This pseudo R-squared statistic wasnot bounded between zero and one in small samples (because the domain isnot precisely the same as for the estimation sample). The ten mostpredictive group transform variables and the ten most predictivecross-product variables were kept in the dataset and passed to the modelestimation routine. Only the top five predictive beta transformvariables were kept. Variables that had low or negative univariate orbivariate pseudo R-squared statistics were also permanently dropped fromthe potential candidate variable pool. These variable transforms arelisted in Table 3, below.

TABLE 3 Variable Transforms Group transforms Cross-products Betatransforms anomaly k-means × anomaly anomaly k-means T2 mean 3 pix. rad× k-means anomaly T2 mean 3 pix. rad. PD 3 pix. prep × T2 mean 3 pix.rad. anomaly T2 med. 3 pix. rad. T2 median 3 pix. rad × T2 med. 3 pix.rad. anomaly PD 3 pixel preprocessing T2 mean 3 pix. rad × pd 3 pix.preproc. k-means T1 mean 5 pix. rad. T2 median 3 pix. rad × k-means T1med. 5 pix. rad. PD 3 pix. prep × k-means T2 med. 5 pix. rad. PD 3 pix.prep × T2 mean 3 pix. rad yellow colorspace T2 med. 3 pix. rad × T2 mean3 pix. rad PD 5 pix. prep. PD 3 pix. prep × T2 med. 3 pix. rad

iv. Model Estimation and Variable Selection.

As with most pattern recognition examples, there were far more potentialcandidate variables for inclusion than could practically be accommodatedin a predictive model, which poses several significant risks. One isthat potentially useful candidates are overlooked simply because thereare too many variables to evaluate. Another is that if a systematicroutine for evaluating and including variables is used, it can lead tooverfitting. Finally, many candidate variables are likely to beredundant, which can cause problems for the estimation routines. Forexample, the mean of the T1 modality was taken over neighborhoodsranging from a 3-pixel radius to a 9-pixel radius, and all were includedas candidate variables.

The RIPNet™ procedure used in this example deals with these risks bycombining the theoretical nonlinear curve fitting capability of thetypical ANN with the stability of hierarchical techniques. This searchover nonlinear combinations and transformations of input variables canthen be used in a standard maximum likelihood logit model. RIPNet™contains algorithms for variable generation (network nodes), variabletesting, and model estimation.

A typical single hidden layer feed-forward network may have two inputsand one output and use so-called squashing (s-shaped) functions. Thesesquashing functions deliver the power of ANNs because they exhibitseveral different behaviors depending on the settings of the parametersβ and γ. Examples of such settings include: inverse; logarithmic;exponential; and threshold functions.

The RIPNet modeling strategy starts with a functional form whoserichness and nonlinearity stem from the functions. Among the keycontributions of the RIPNet algorithm is a high-yield method forgenerating simulated network nodes. In spite of its outward simplicity,this form of model can be used to closely approximate the performance oftraditional ANNs.

Node selection within a class of relatively tractable models is the nextstep in the process. As with the variable pre-filtering steps, nodeselection was based upon the use of a validation sample to checkout-of-sample performance. Candidate nodes were entered into the modelin order of their validated prediction performance in a predictivemodel. Redundancy was handled in two ways within the selectionprocedure. First, as additional nodes were entered into the model, theyare orthogonalized so as to remove redundant components. Multiplethresholds were tested so that an optimal level of node orthogonalitycould be identified. Second, a threshold for redundancy was picked suchthat only nodes with less than 5% of their variance explained by othernodes in the model could be entered.

v. Model Validation.

A final model validation step was used to ensure that whichever modelwas selected as the final model, it was better than a simpler benchmarkor other candidate models. Data Miner's Reality Check™ (DMRC) was againused to test the models in this way. This technique utilized out ofsample predictions and bootstrap distributions to generate validp-values for the hypothesis that the tested model had the sameperformance as the benchmark model. Low p-values indicated that thetested model exhibited significantly better performance.

vi. Results.

The predictive models described herein have significantly improvedpredictive performance relative to the leading techniques in use today.FIG. 9 summarizes the performance of the models described above. In thefigure, Table A summarizes the performance of the RIPNet models based ontwo different statistical measures: the maximum Kolmogorov-Smirnovstatistic; and the Gini coefficient, each of which measure aspects ofthe ROC (regional operational characteristics) curve. Models based onthe K-means approach are used as a basis for comparison. These resultsdemonstrate that the RIPNet models universally perform between 25% and30% better in absolute terms than K-means. This translates to a 50%higher true positive rate a given level of false positives. The ROCcurves from which these results were derived are shown in FIG. 9.

In addition to statistical performance measures, the combined modelresults shown in image form (FIG. 8) also support this conclusion. InFIG. 8, for each image the labeled ground-truth is presented in the leftpanel, and the modeling predictions for the same image are presented inthe right panel. In the figure, muscle appears pink in the labeledimages, and red in the model results. Lipids appear white in the labeledimages, and blue in the model results, while plaque appears yellow inboth images. Each pixel was assigned to a category depending on whichmodel generated the highest probability for that pixel. Pixels withbelow 30% probability for all of the model predictions were coded asblank.

The image developed from the predictive model shown in FIG. 8A providesa clear example of the capabilities of the predictive models of theinvention. In particular, there is a high degree of correspondencebetween the pixels labeled plaque in the original, ground-truth, labeledimage, and those labeled plaque by the computational models. Likewise,muscle was also well identified. The image in FIG. 8B demonstrates theability of the models of the invention to detect not only the hardvascular plaques, but also plaques having lipid components. The muscleareas are not uniformly identified, but this occurs in precisely thoseareas where the original image is plagued by artifacts and where themuscle wall is thin. Interestingly, there are some false-positivescoming from the muscle model along areas of the fibrous cap enclosingthe lipid core shown in this image.

Over 400 variables were included in this analysis, and thousands ofnetwork nodes were created from these variables. Out of all of thesenodes, 140 were selected for each model. This would normally beconsidered a large number, but for the large number of observations inthe datasets. Table 4, below, illustrate the top several nodes selectedfor each model. An examination of these nodes illustrates the benefitsof using an automated technique over other approaches. For example, theplaque model contains mainly T1 and PD variables, and for almost all ofthe included variables a 7-pixel neighborhood measure was selected overall of the others available. These types of selections would have beenalmost impossible to reproduce manually without enormous effort.Likewise, many of the combinations of variables are not obvious to thehuman eye. For example, the top node for the lipid model is a linearcombination of the maximum of the four-pixel neighborhood for T2, thediscrete cosine transform of PD, and the anomaly variable—combinationsfor which no clear explanation exists today. Certainly none of the usualheuristic methods would have uncovered these.

TABLE 4 Top Relevant Inputs Rank Type Description Plaque Model 1 NodeKmeans 2 Node T1 Mean of 7 pix rad. 3 Node PD Median of 7 pix rad 4 NodeT1 Min of 3 pix rad 5 Node T1 Min of 7 pix rad 6 Node PD DCT of 7 pixrad Muscle Model 1 Node Anomaly 2 Node Anomaly T2 Var. 4 pix rad 3 NodeAnomaly PD Skew. 3 pix rad 4 Hidden T1 Pre-proc. 3 pix rad Unit T2Pre-proc. 5 pix rad CR PD DCT of 7 pix rad Anomaly variable Lipid Model1 Hidden T2 Max of 4 pix rad Unit PD DCT 5 pix rad Anomaly 2 Hidden T1Mean of 3 pix rad Unit PD Max of 3 pix rad Yellow - CMYK transform PDDCT of 5 pix rad Anomaly variable 3 Hidden T2 Range of 3 pix rad Unit PDVariance of 4 pix rad PD IQR of 5 pix rad PD DCT of 6 pix rad Anomaly

It is important to note also the highly non-linear nature of thesemodels. The fact that the anomaly variable appears as one of the mostpredictive variables serves to underscore this fact. Hidden units, thegroup transforms, the beta transforms, etc. are all non-lineartransformations of the inputs that appear as top-ranked variables inthese models.

To avoid overfit, precautions were taken. Testing was performed onseveral images that were reserved entirely from the modeling process. Asshown in FIG. 10, the models do a reasonably good job separating musclefrom plaque. The muscle model tracks the general outline of the arterywall, and the model identifies the labeled plaque areas well. As thisimage was challenging for expert radiologists to label in the firstplace, it made for a challenging, and ultimately successful, test.

Example 4 3-D Blood Vessel Models

This example describes a preferred method for generatingthree-dimensional models of blood vessels that have been imaged using amedical imaging instrument. Specifically, FIG. 13 shows a 3-D renderingof a carotid artery in the area of the carotid bifurcation. In thefigure, the inner arterial wall (1920) represents the boundary of thelumen. Plaque (1910) resides between the inner surface of the arterialwall and the exterior surface of the artery (not shown). This model wasderived from eleven in vivo MRI tissue slices using only the T1 mode. Inorder to generate the model, the following steps were used transform thetissue slice images. Initial, the data for each slice was passed througha low pass filter (e.g., adaptive Wiener filter). The location of thelumen center for each slice was estimated based on the position of thelumen centroid from the preceding slice. Each image was then croppedafter centering on the estimated lumen location. A linear search ofthreshold intensities was then performed to reveal lumen area close tothe estimated centroid. After verifying that the lumen had the requisitemorphological features, including area and eccentricity, the position ofthe lumen centroid was re-estimated. Tissue segmentation was thenperformed to identify lipid features near the lumen centroid.

As will be appreciated, the foregoing process was modified slightlydepending on whether the slice was above or below the carotidbifurcation. When the algorithm was tracking two lumens (i.e., in slicesabove the carotid bifurcation), the geometric mean of the two lumencentroids were used. The resulting slices were then re-centered in orderto compensate for axial misalignment. The addition of other tissueinformation, including that for muscle, adventitia, and plaquecomponents such as lipid, hemorrhage, fibrous plaque, and calcium, canalso be included. The resulting models will allow for visualization andautomated quantification of plaque size, volume, and composition.

All of the processes, systems, and articles of manufacture described andclaimed herein can be made and executed without undue experimentation inlight of this specification. While the methods, systems, and computerprogram products of the invention have been described in terms ofpreferred embodiments and optional features, it will be apparent tothose of skill in the art that modifications and variations may beapplied to the methods and in the steps or in the sequence of steps ofthe methods described herein without departing from the spirit and scopeof the invention. More specifically, it will be apparent that differentalgorithms, software, and data can be adapted for the automateddetection and analysis of vascular plaque. All such equivalent orsimilar adaptations, embellishments, modifications, and substitutesapparent to those skilled in the art are deemed to be within the spiritand scope of the invention as defined by the appended claims.

The invention has been described broadly and generically herein. Each ofthe narrower species and subgeneric groupings falling within the genericdisclosure also form part of the invention. This includes the genericdescription of the invention with a proviso or negative limitationremoving any subject matter from the genus, regardless of whether or notthe excised material is specifically recited herein.

The invention illustratively described herein suitably may be practicedin the absence of any element(s) not specifically disclosed herein asessential. The terms and expressions which have been employed are usedas terms of description and not of limitation, and there is no intentionthat in the use of such terms and expressions of excluding anynow-existing or later-developed equivalents of the features shown anddescribed or portions thereof, but it is recognized that variousmodifications are possible within the scope of the invention claimed.Also, the terms “comprising”, “including”, “containing”, etc. are to beread expansively and without limitation. It must be noted that as usedherein and in the appended claims, the singular forms “a”, “an”, and“the” include plural reference unless the context clearly dictatesotherwise.

All patents, patent applications, and publications mentioned in thisspecification are indicative of the levels of those of ordinary skill inthe art to which the invention pertains. All patents, patentapplications, and publications are herein incorporated by reference intheir entirety for all purposes and to the same extent as if eachindividual patent, patent application, or publication was specificallyand individually indicated as being incorporated by reference.

1. A fully automated method of classifying plaque components todetermine whether a blood vessel contains plaque, the method comprisingcomputationally processing at least a first processable data type and asecond processable data type obtained using at least one non-invasivemedical imaging system in order to analyze at least one cross section ofa blood vessel of a patient's vasculature using a plurality of storedtissue classifier elements developed using statistical modeling todetermine if the blood vessel comprises at least one tissue correlatedwith the presence of plaque, in which event the blood vessel isdetermined to contain plaque, wherein the fully automated method doesnot require human intervention and wherein at least one of the first andsecond processable data types is processable magnetic resonance datagenerated by an MRI instrument.
 2. A fully automated method according toclaim 1 wherein the blood vessel comprises a portion of the vasculaturesupplying blood to an organ selected from the group consisting of abrain and a heart, wherein when the brain is the organ the blood vesselis a carotid artery and wherein when the heart is the organ the bloodvessel is a coronary artery, further wherein the patient is human.
 3. Afully automated method according to claim 1 wherein the processable dataare generated by pre-processing raw data generated by the medicalimaging system.
 4. A fully automated method according to claim 3 furthercomprising normalizing the processable data prior to computationallyprocessing the processable data.
 5. A fully automated method accordingto claim 1 wherein the plurality of stored tissue classifier elementsare developed using known outcome data by a process selected from thegroup consisting of logistic regression, decision trees, non-parametricregression, Fisher discriminant analysis, Bayesian network modeling, anda fuzzy logic system.
 6. A fully automated method according to claim 5wherein at least one of the plurality of stored tissue classifierelements is determined by a process selected from the group consistingof post-operative histological examination, direct tissue inspection,and labeling by one or more experts.
 7. A fully automated methodaccording to claim 1 further comprising at least one of the following:a. computationally processing the processable data to determine whetherthe blood vessel, in the region of the cross section, further comprisesat least one tissue selected from the group consisting of adventitia, acalcium deposit, a cholesterol deposit, fibrous plaque, and thrombus; b.computationally processing the processable magnetic resonance data intoregistration, wherein the registration is accomplished by aligningcomponents derived from the processable magnetic resonance data about arepresentation that represents a landmark selected from the groupconsisting of a physical landmark and a computational landmark, whereinthe physical landmark is a vessel branch point vessel and thecomputational landmark is a lumen centroid; c. computationallyprocessing processable data of a plurality of spaced cross sections ofthe blood vessel; d. computationally processing processable data of aplurality of spaced cross sections of the blood vessel andcomputationally rendering a three-dimensional model of the blood vesselover at least a portion of a region bounded by most distantly spacedcross sections of the blood vessel; e. generating an output filecomprising data resulting from the computationally processing, whereinthe output file comprises a computationally rendered three-dimensionalmodel of the blood vessel over at least a portion of a region bounded bymost distantly spaced cross sections of the blood vessel; f.computationally processing processable data of a plurality of spacedcross sections of the blood vessel, computationally rendering athree-dimensional model of the blood vessel over at least a portion of aregion bounded by most distantly spaced cross sections of the bloodvessel, and computationally determining plaque volume present in thethree-dimensional model of the blood vessel; g. computationallyprocessing processable data of a plurality of spaced cross sections ofthe blood vessel, computationally rendering a three-dimensional model ofthe blood vessel over at least a portion of a region bounded by mostdistantly spaced cross sections of the blood vessel, and computationallydetermining composition of plaque present in the three-dimensional modelof the blood vessel; and h. computationally processing processable dataof a plurality of spaced cross sections of the blood vessel,computationally rendering a three-dimensional model of the blood vesselover at least a portion of a region bounded by most distantly spacedcross sections of the blood vessel, and computationally determiningcomposition of plaque present in the three-dimensional model of theblood vessel and distinguishing whether the plaque is a vulnerableplaque or a stable plaque.
 8. A fully automated method of assessingeffectiveness of a therapeutic regimen, comprising: a. determining aplaque volume in at least a portion of a blood vessel of a patient usinga fully automated method according to claim 1; b. delivering to thepatient a therapeutic regimen comprising administration of a drugexpected to stabilize or reduce plaque volume over the course of thetherapeutic regimen; and c. during and/or at the end of the therapeuticregimen determining whether the plaque volume has stabilized or beenreduced, thereby allowing assessment of the effectiveness of thetherapeutic regimen.
 9. A fully automated method for determining whethera blood vessel of a patient contains plaque, comprising: a. obtainingprocessable data of at least one cross section of a blood vessel of apatient's vasculature, wherein the processable data are derived from rawdata collected using a non-invasive medical imaging system thatcomprises an MRI instrument, wherein the processable data comprises atleast a first processable magnetic resonance data type and a secondprocessable magnetic resonance data type generated by an MRI instrument;and b. communicating the processable data to a computer configured toreceive and computationally process the processable data usingstatistical classifiers developed using statistical modeling todetermine whether the blood vessel in a region of the cross section(s)comprises at least one tissue correlated with the presence of plaque, inwhich event the blood vessel is determined to contain plaque; whereinthe computational processing and determination of whether the bloodvessel contains plaque does not require human intervention.
 10. A fullyautomated method according to claim 9 wherein the blood vessel comprisesa portion of the vasculature supplying blood to an organ selected fromthe group consisting of a brain and a heart, wherein when the brain isthe organ the blood vessel is a carotid artery and wherein when theheart is the organ the blood vessel is a coronary artery, furtherwherein the patient is human.
 11. A fully automated method according toclaim 9 wherein the non-invasive medical imaging system and the computerare located at different locations.
 12. A fully automated methodaccording to claim 9 wherein the computer resides in a computationalcenter physically removed from each of a plurality of imaging centers,each of which imaging centers comprises a non-invasive medical imagingsystem capable of generating raw magnetic resonance data from whichprocessable magnetic resonance data can be derived, wherein at least oneof the imaging centers communicates raw data to the computational centervia a telecommunications link.
 13. A fully automated method according toclaim 9 further comprising communicating results of the computationalprocessing and determination of whether the blood vessel contains plagueto an address specified as being affiliated with the non-invasivemedical imaging system used to collect the raw data.